<html>
<head><meta charset="utf-8"><title>Safe offset_from? · t-libs · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/index.html">t-libs</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html">Safe offset_from?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="246055839"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246055839" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Simon Sapin <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246055839">(Jul 15 2021 at 06:16)</a>:</h4>
<p><a href="https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.offset_from">https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.offset_from</a> has pretty steep safety requirements. Is it possible to build a safe API like it that works on slices and is fallible? (returns <code>None</code> if the parameter is not a sub-slice)</p>



<a name="246055947"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246055947" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Simon Sapin <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246055947">(Jul 15 2021 at 06:19)</a>:</h4>
<p>In particular: how to check whether pointers are from the same allocated object, without the check itself invoking provenance daemons?</p>



<a name="246056753"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246056753" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246056753">(Jul 15 2021 at 06:35)</a>:</h4>
<p>You can't. It is a compiler concept, not something physically represented after compilation.</p>



<a name="246059532"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246059532" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Simon Sapin <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246059532">(Jul 15 2021 at 07:18)</a>:</h4>
<p>I ended up doing <code>usize</code> arithmetic without using <code>ptr::offset_from</code>. Does this look correct?</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="w">        </span><span class="k">fn</span> <span class="nf">address_range</span><span class="p">(</span><span class="n">slice</span>: <span class="kp">&amp;</span><span class="p">[</span><span class="kt">u8</span><span class="p">])</span><span class="w"> </span>-&gt; <span class="nc">std</span>::<span class="n">ops</span>::<span class="n">RangeInclusive</span><span class="o">&lt;</span><span class="kt">usize</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="kd">let</span><span class="w"> </span><span class="n">start</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">slice</span><span class="p">.</span><span class="n">as_ptr</span><span class="p">()</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="kt">usize</span><span class="p">;</span><span class="w"></span>
<span class="w">            </span><span class="kd">let</span><span class="w"> </span><span class="n">end</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">start</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">slice</span><span class="p">.</span><span class="n">len</span><span class="p">();</span><span class="w"></span>
<span class="w">            </span><span class="n">start</span><span class="o">..=</span><span class="n">end</span><span class="w"></span>
<span class="w">        </span><span class="p">}</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">slice_addresses</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">address_range</span><span class="p">(</span><span class="n">slice</span><span class="p">.</span><span class="n">as_bytes</span><span class="p">());</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">on_disk_addresses</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">address_range</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">dirstate_map</span><span class="p">.</span><span class="n">on_disk</span><span class="p">);</span><span class="w"></span>
<span class="w">        </span><span class="k">if</span><span class="w"> </span><span class="n">on_disk_addresses</span><span class="p">.</span><span class="n">contains</span><span class="p">(</span><span class="n">slice_addresses</span><span class="p">.</span><span class="n">start</span><span class="p">())</span><span class="w"></span>
<span class="w">            </span><span class="o">&amp;&amp;</span><span class="w"> </span><span class="n">on_disk_addresses</span><span class="p">.</span><span class="n">contains</span><span class="p">(</span><span class="n">slice_addresses</span><span class="p">.</span><span class="n">end</span><span class="p">())</span><span class="w"></span>
<span class="w">        </span><span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="kd">let</span><span class="w"> </span><span class="n">offset</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">slice_addresses</span><span class="p">.</span><span class="n">start</span><span class="p">()</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">on_disk_addresses</span><span class="p">.</span><span class="n">start</span><span class="p">();</span><span class="w"></span>
<span class="w">            </span><span class="nb">Some</span><span class="p">(</span><span class="n">offset_from_usize</span><span class="p">(</span><span class="n">offset</span><span class="p">))</span><span class="w"></span>
<span class="w">        </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="nb">None</span><span class="w"></span>
<span class="w">        </span><span class="p">}</span><span class="w"></span>
</code></pre></div>



<a name="246061716"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246061716" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246061716">(Jul 15 2021 at 07:47)</a>:</h4>
<p>It matters what offset_from_usize is offsetting from. If it's working off <code>on_disk_address</code> then this seems right to me</p>



<a name="246061815"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246061815" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246061815">(Jul 15 2021 at 07:48)</a>:</h4>
<p>you will end up with a pointer with the <code>on_disk_address</code>'s provenance. And since that object is strict superset of <code>slice</code> you aren't at risk of making an invalid pointer.</p>



<a name="246066164"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246066164" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Simon Sapin <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246066164">(Jul 15 2021 at 08:34)</a>:</h4>
<p>(Sorry I didn’t remove the parts less relevant, <code>offset_from_usize</code> does not offset pointers but returns a <code>struct Offset([u8; 4])</code> for 32-bit big-endian without alignment, for writing in a binary file format)</p>



<a name="246066521"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246066521" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Simon Sapin <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246066521">(Jul 15 2021 at 08:38)</a>:</h4>
<p>Why does <code>ptr::offset_from</code> have so much safety requirements compared to converting to <code>usize</code> and subtracting?</p>



<a name="246072543"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246072543" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jacob Lifshay <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246072543">(Jul 15 2021 at 09:47)</a>:</h4>
<p><span class="user-mention silent" data-user-id="219747">Simon Sapin</span> <a href="#narrow/stream/219381-t-libs/topic/Safe.20offset_from.3F/near/246066521">said</a>:</p>
<blockquote>
<p>Why does <code>ptr::offset_from</code> have so much safety requirements compared to converting to <code>usize</code> and subtracting?</p>
</blockquote>
<p>I'd expect the semantics are inherited from C. Part of requiring the offset to be divisible by <code>size_of::&lt;T&gt;()</code> is so the compiler can use fancy math to convert the division into a shift and/or multiplication.</p>



<a name="246076753"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246076753" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246076753">(Jul 15 2021 at 10:37)</a>:</h4>
<p>It preserves provenance of the object being offset from, which gives the compiler information about what the object ought to be.</p>



<a name="246076850"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246076850" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246076850">(Jul 15 2021 at 10:38)</a>:</h4>
<p>i.e. the operation of offsetting from a pointer to an object that compiler otherwise might not know anything about gives it useful information about what the object may be (e.g. its size)</p>



<a name="246076953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246076953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246076953">(Jul 15 2021 at 10:39)</a>:</h4>
<p>You can use <code>wrapping_offset</code> for a less strong variant.</p>



<a name="246077036"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246077036" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246077036">(Jul 15 2021 at 10:40)</a>:</h4>
<p>casting to an integer is the most flexible and most optimization defeating option.</p>



<a name="246077107"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246077107" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246077107">(Jul 15 2021 at 10:41)</a>:</h4>
<p>Ah, I guess we don't have a <code>wrapping_offset_from</code> <span aria-label="frown" class="emoji emoji-1f641" role="img" title="frown">:frown:</span></p>



<a name="246627999"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/246627999" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#246627999">(Jul 20 2021 at 18:07)</a>:</h4>
<p>There used to be, but it was deprecated a year and a bit ago: <a href="https://github.com/rust-lang/rust/pull/73580">https://github.com/rust-lang/rust/pull/73580</a></p>



<a name="248378779"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/248378779" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Simon Sapin <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#248378779">(Aug 04 2021 at 17:08)</a>:</h4>
<p><code>offset_from</code> returns <code>isize</code>, not another pointer. What does preserving provenance even mean there?</p>



<a name="248379334"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/248379334" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#248379334">(Aug 04 2021 at 17:13)</a>:</h4>
<p>provenance is not a feature of a type, but of a value, but you are right, that returned <code>isize</code> has no provenance if used on two pointers with the same provenance, you need a pointer with provenance in order to obtain a real pointer again. Using two pointers with different provenance as arguments to <code>offset_from</code> is UB as we don't have the concept of values with two separate provenances, and thus any behaviour with such values is undefined</p>



<a name="248380591"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/248380591" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Simon Sapin <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#248380591">(Aug 04 2021 at 17:23)</a>:</h4>
<p>Why is it useful to declare UB when <code>offset_from</code> arguments have different provenance, given that the return value does not have any provenance itself?</p>



<a name="248385511"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/248385511" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#248385511">(Aug 04 2021 at 18:02)</a>:</h4>
<p>the return value will have <em>two</em> provenances if you pass two args of different provenance, and we haven't defined what the meaning of two provenances is. There's probably more subtlety here, but this is as far as my understanding goes</p>



<a name="248385550"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/Safe%20offset_from%3F/near/248385550" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/Safe.20offset_from.3F.html#248385550">(Aug 04 2021 at 18:03)</a>:</h4>
<p>if you pass two arguments with the same provenance, the return value is indeed provenance-free</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>